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Abstract 

Background: There is continuing interest among practitioners, policymakers and researchers in the evaluation of 
complex interventions stemming from the need to further develop the evidence base on the effectiveness of 
healthcare and public health interventions, and an awareness that evaluation becomes more challenging if 
interventions are complex. 

We undertook an analysis of published journal articles in order to identify aspects of complexity described by 
writers, the fields in which complex interventions are being evaluated and the challenges experienced in design, 
implementation and evaluation. This paper outlines the findings of this documentary analysis. 

Methods: The PubMed electronic database was searched for the ten year period, January 2002 to December 201 1, 
using the term "complex intervention*" in the title and/or abstract of a paper. We extracted text from papers to a 
table and carried out a thematic analysis to identify authors' descriptions of challenges faced in developing, 
implementing and evaluating complex interventions. 

Results: The search resulted in a sample of 221 papers of which full text of 216 was obtained and 207 were 
included in the analysis. The 207 papers broadly cover clinical, public health and methodological topics. Challenges 
described included the content and standardisation of interventions, the impact of the people involved (staff and 
patients), the organisational context of implementation, the development of outcome measures, and evaluation. 

Conclusions: Our analysis of these papers suggests that more detailed reporting of information on outcomes, 
context and intervention is required for complex interventions. Future revisions to reporting guidelines for both 
primary and secondary research may need to take aspects of complexity into account to enhance their value to 
both researchers and users of research. 



Background 

There is continuing interest among practitioners, 
policymakers and researchers in the evaluation of com- 
plex interventions. This interest stems from the need 
to further develop the evidence base on the effective- 
ness of healthcare and public health interventions, and 
an awareness that evaluation becomes more challen- 
ging as interventions move along the spectrum from 
'simple' towards more complex interventions [1]. This 
focus on complexity is also driven by ongoing debate 
about the most appropriate methods for evaluating 
health systems, and the recognition that it is important 
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to know not just whether health system interventions 
'work', but also about when, why, how and in what 
circumstances such interventions work well [2,3]. 

A further stimulus has been the Medical Research 
Council's (MRC) 'A framework for development and 
evaluation of RCTs for complex interventions to im- 
prove health^ originally published in 2000 [4] and revised 
and extended in 2008 [5]. This guidance was published 
in response to the difficulties faced by those attempting 
to develop complex interventions and evaluate their im- 
pact. It describes complex interventions as being 'built 
up from a number of components, which may act both 
independently and inter-dependently' [4]. These compo- 
nents include behaviours, behaviour parameters and 
methods of organising those behaviours, and they may 
have an effect at individual patient level, organisational 
or service level or population level (or all of these in 
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some cases). The MRC's 2008 guidance also emphasises 
the numbers of components and their interactions, behav- 
iours, organisational levels and outcomes, and goes further 
than the 2000 framework in outlining the variability of de- 
sired outcomes and the degree to which flexibility or tai- 
loring of the intervention is permitted. Both documents 
highlight the importance of establishing both whether an 
intervention is effective and how it works. 

The term 'complex intervention' is now used exten- 
sively in the academic health literature to describe both 
health service and public health interventions. Complex 
interventions have been the topic of numerous confer- 
ences and meetings, the focus of funding calls, and will 
be the subject of a new chapter in the Cochrane Hand- 
book for Systematic Reviews of Interventions [6]. The 
common usage of the term indicates increasing recogni- 
tion of complexity and its implications for the develop- 
ment and evaluation of interventions. It may also be the 
case that, as the term has achieved wider application, it 
has come to be used strategically by researchers to add 
authority and currency to funding proposals and aca- 
demic articles. However, it is not always clear that 'com- 
plexity' is being used to refer to the same things, nor 
what measures researchers are taking to evaluate it. It 
has been suggested, for example, that what is described 
as 'complexity' is actually just 'complicatedness' - a very 
different concept [7]. 

We undertook an analysis of published journal articles 
in the field of health in which complexity was an import- 
ant element. Our aim was to identify the aspects of com- 
plexity described by writers; the fields in which complex 
interventions are being evaluated; and to describe chal- 
lenges experienced due to the complexity of inter- 
ventions and how authors dealt with these. This paper 
outlines the findings of this documentary analysis focus- 
ing, in particular, on the challenges of designing, 
implementing and evaluating complex interventions 
described by authors. 

Methods 

Search strategy 

The PubMed electronic database was searched for jour- 
nal articles published in the ten year period, January 
2002 to December 2011. The start date was chosen to 
allow enough time for papers referring to the MRC guid- 
ance (2000) to have been published. The search identi- 
fied the term "complex intervention*" in the title and/or 
abstract of a paper, excluding papers not written in Eng- 
lish. Research reports, trial protocols, systematic reviews, 
meta-analyses, discussion pieces, published oral presen- 
tations and letters were included. We then undertook a 
content analysis of the papers to identify authors' de- 
scriptions of challenges faced in developing, imple- 
menting and evaluating complex interventions. The 



search was undertaken systematically as described above 
but we did not conduct a critical analysis of each paper 
as our principal aim was to provide a snapshot of 
current practice rather than a comprehensive review. 

Analysis 

Having read the papers, we extracted text to a table from 
each paper. Columns included title, author, study topic 
(e.g. clinical, public health, etc.), definitions of complex 
interventions used by authors, problems identified by 
authors (using the search terms: challenge, barrier, 
difficult, limit to identify difficulties described), and cited 
literature on complex interventions. 

The process of analysing the papers' content and iden- 
tifying challenges described by authors produced a num- 
ber of themes which we used to structure the results 
section. These are: intervention design (descriptions of 
challenges derived from the nature or content of the 
intervention); intervention implementation (challenges 
in implementing complex interventions); contextual 
characteristics (aspects of context that may influence im- 
plementation or evaluation of complex interventions); 
outcomes (reflecting the difficulties posed by the out- 
comes of complex interventions) and evaluation (describ- 
ing challenges to evaluation). We are not suggesting that 
these themes are mutually exclusive. The design and 
content of an intervention, for example, are influenced by 
the context in which it will be implemented and the 
methodology used to evaluate it. Quotes from papers were 
selected to illustrate issues raised by authors. 

Results 

The search resulted in a sample of 221 papers of which 
full text of 216 (98%) was obtained and 207 were in- 
cluded in the analysis. Nine papers were excluded be- 
cause their subject matter was not relevant for our 
purposes. A small number of the papers included were 
published online in 2011 but in print in 2012. 

The 207 papers broadly cover clinical (45%), methodo- 
logical (27%), health promotion (23%) and public health 
(3%) topics with a small number of 'others' (1%). All 
those included in the analysis are listed in Table 1. Some 
papers focus on particular health conditions, such as 
cancer, diabetes, HIV and mental illness; some on health 
and social interventions, including palliative care ser- 
vices, complementary therapies and decision aids; and 
others on methodological and theoretical issues such as 
causality, the use of normalisation process theory, and 
approaches to health promotion. 

Use of MRC guidance 

As noted above, MRC guidance on the development and 
evaluation of complex interventions [4,5] has been avail- 
able since 2000. Without making assumptions about 
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Table 1 Papers analysed by year of publication 


2002 


Loeb MB Application of the development stages of a cluster randomized trial to a framework for valuating complex 


Methodology 


health interventions. BMC Health Serv Res 2002 2:13 




Muhlhauser I, Berger, M. Patient education - evaluation of a complex intervention. Diabetologia 2002 45:1 723-1 733 


Health 




promotion 


Nazareth I, Freemantle N, Duggan C, Mason J, Haines, A. Evaluation of a complex intervention for changing professional 


Health 


behaviour: the Evidence Based Out Reach (EBOR) Trial. J Health Serv Res Policy 2002 7:230-238 


promotion 


Treweek SP, Glenton C, Oxman AD, Penrose A Computer-generated patient education materials: do they affect professional 


Health 


practice? A systematic review. 2002 J Am Med Inform Assoc 4:346-358 


promotion 


2003 


Emery JD. Effect of computerised evidence based guidelines. BMJ 2003 326:394 


Methodology 


Glazener CM, Evans JH, Peto RE. Tricyclic and related drugs for nocturnal enuresis in children. Cochrane Database Syst Rev 2003 


Clinical 


3:CD002117 




Lyles JS, Hodges A, Collins C, Lein C, Given CW, Given B, D'Mello D, Osborn GG, Goddeeris J, Gardiner JC, Smith RC. Using nurse 


Clinical 


practitioners to implement an intervention in primary care for high-utilizing patients with medically unexplained 




symptoms. Gen Hosp Psychiatry 2003 25:63-73 




Omar R. The evidence for prosthodontic treatment planning for older, partially dentate patients. Med Princ Pract 2003 12 


Clinical 


Suppl 1:33-42 




Rollnick S. General practitioner screening for excessive alcohol use. Paper enables open debate about a complex 


Health 


intervention. BMJ 2003 326:336 


promotion 


2004 


Byng R, Jones R. Mental Health Link: the development and formative evaluation of a complex intervention to improve 


Methodology 


shared care for patients with long-term mental illness. J Eval Clin Pract 2004 10:27-36 




Conway TL, Woodruff SI, Edwards CC, Hovell MF, Klein J. Intervention to reduce environmental tobacco smoke exposure in 


Health 


Latino children: null effects on hair biomarkers and parent reports, fob Control 2004 1 3:90-92 


promotion 


Drescher U, Warren F, Norton K. Towards evidence-based practice in medical training: making evaluations more meaningful. 


Methodology 


Med E due 2004 1 2: 1 288- 1 294 




Figar S, Waisman G, De Quiros FG, Galarza C, Marchetti M, Loria GR, Camera L, Seinhart D, Camera M. Narrowing the gap in 


Clinical 


hypertension: effectiveness of a complex antihypertensive program in the elderly. 




Dis Manag 2004 7:235-243 




Glazener CM, Evans JH, Peto RE. Complex behavioural and educational interventions for nocturnal enuresis in children. 


Health 


Cochrane Database Syst Rev 2004 1:CD004668 


promotion 


Gulmezoglu AM, Villar J, Grimshaw J, Piaggio G, Lumbiganon P, Langer A. Cluster randomized trial of an active, multifaceted 


Other 


information dissemination intervention based on The WHO Reproductive health library to change obstetric practices: 




methods and design issues [ISRCTN 14055385] BMC Med Res Methodol 2004 4:2 




Hammond A, Young A, Kidao R. A randomised controlled trial of occupational therapy for people with early rheumatoid 


Clinical 


arthritis. Ann Rheum Dis 2004 63:23-30 




Lawson H The logic of collaboration in education and the human services. J Interprof Care 2004 18: 225-237 


Other 


Paterson C, Britten N. Acupuncture as a complex intervention: a holistic model. J Altern Complement Med 2004 10: 791-801 


Clinical 


Power R, Langhaug LF, Nyamurera T, Wilson D, Bassett MT, Cowan FM. Developing complex interventions for rigorous 


Methodology 


evaluation - a case study from rural Zimbabwe. Health Educ Res 2004 19:570-575 




Shemilt 1, Harvey 1, Shepstone L, Swift L, Reading R, Mugford M, Belderson P, Norris N, Thoburn J, Robinson JA national evaluation 


Health 


of school breakfast clubs: evidence from a cluster randomized controlled trial and an observational analysis. Child Care 


promotion 


Health Dev 2004 30: 41 3-427 




Smith S, Bury, O'Leary M, Shannon W, Tynan A, Staines A, Thompson C. The North Dublin randomized controlled trial of 


Clinical 


structured diabetes shared care. Earn Pract 2004 21: 39-45 




Whitford DL, Roberts SH, Griffin S. Sustainability and effectiveness of comprehensive diabetes care to a district population. 


Clinical 


Diabet Med 2004 21: 1221-1228. 




2005 


Bonetti D, Eccles M, Johnston M, Steen N, Grimshaw J, Baker R, Walker A, Pitts N. Guiding the design and selection of 


Methodology 


interventions to influence the implementation of evidence-based practice: an experimental simulation of a complex 




intervention trial. Soc Sci Med 2005 60:2135-2147 




Eldridge S, Spencer A, Cryer C, Parsons S, Underwood M, Feder G. Why modelling a complex intervention is an important 


Methodology 


precursor to trial design: lessons from studying an intervention to reduce falls-related injuries in older people. J Health 




Serv Res Policy 2005 1 0: 1 33- 1 42 
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Table 1 Papers analysed by year of publication (Continued) 



Greenhalgh T, Collard A, Begum N. Sharing stories: complex intervention for diabetes education in minority ethnic groups Health 
who do not speak English. BMJ 2005 330:628 promotion 

Levesque L, .Guilbault G, Delormier T, Potvin L, Unpacking the black box: a deconstruction of the programming approach Health 

and physical activity interventions implemented in the Kahnawake Schools Diabetes Prevention Project. Health Promot promotion 
Pract 2005 6:64-71 

Nilsson S, Spak F, Marklund B, Baigi A, Allebeck P. Attitudes and behaviours with regards to androgenic anabolic steroids Other 
among male adolescents in a county of Sweden. Subst Use Misuse 2005 40:1-12 

Robinson L, Francis J, James P, Tindle N, Greenwell K, Rodgers H. Caring for carers of people with stroke: developing a Health 

complex intervention following the Medical Research Council framework. Clin Rehabil 2005 19:560-571 promotion 

Rowlands G, Sims J, Kerry S. A lesson learnt: the importance of modelling in randomized controlled trials for complex Methodology 
interventions in primary care. Fam Pract 2005 22:132-139. 

2006 

Blackwood B. Methodological issues in evaluating complex healthcare interventions. J Adv Nurs 2006 54:61 2-622 Methodology 

Bower P, Gilbody S, Richards D, Fletcher J, Sutton A. Collaborative care for depression in primary care. Making sense of a Clinical 
complex intervention: systematic review and meta-regression. Br J Psychiatry 2006 189:484-493 

Byrne M, Cupples ME, Smith SM, Leathern C, Corrigan M, Byrne MC, Murphy AW. Development of a complex intervention for Clinical 
secondary prevention of coronary heart disease in primary care using the UK Medical Research Council framework. 

Am J Manag Care 2006 12:261-266 

Corrigan M, Cupples ME, Smith SM, Byrne M, Leathern CS, Clerkin P, Murphy AW. The contribution of qualitative research in Methodology 
designing a complex intervention for secondary prevention of coronary heart disease in two different healthcare systems. 

BMC Health Serv Res 2006 6:90 

Cullen W, Stanley J, Langton D, Kelly Y, Staines, A, Bury G. Hepatitis C infection among injecting drug users in general practice: Clinical 
a cluster randomised controlled trial of clinical guidelines' implementation. Br J Gen Pract 2006 56:848-856 

Dornan T, Littlewood S, Margolis SA, Scherpbier A, Spencer J, Ypinazar V. How can experience in clinical and community Health 
settings contribute to early medical education? A BEME systematic review. Med Teach 2006 28:3-18 promotion 

Greaves CJ, Farbus L Effects of creative and social activity on the health and well-being of socially isolated older people: Public health 
outcomes from a multi-method observational study. J R Soc Promot Health 2006 126:134-142 

Heaven B, Murtagh M, Rapley T, May C, Graham R, Kaner E, Thomson R. Patients or research subjects? A qualitative study of Methodology 
participation in a randomised controlled trial of a complex intervention. Patient Educ Couns 2006 62:260-270 

Holbrook AM, Thabane L, Shcherbatykh IY, O'Reilly D. E-health interventions as complex interventions: improving the quality Methodology 
of methods of assessment. AMIA Annu Symp Proc 2006 AMIA 2006 Symposium Proceedings: 952 

Jansen YJ, Bal R, Bruijnzeels M, Foets M, Frenken R, de Bont A. Coping with methodological dilemmas; about establishing the Methodology 
effectiveness of interventions in routine medical practice. BMC Health Serv Res 2006 6:160 

Kasper J, Kopke S, Muhlhauser I, Heesen C. Evidence-based patient information about treatment of multiple sclerosis a phase Health 
one study on comprehension and emotional responses. Patient Educ Couns 2006 62:56-63 promotion 

Kunz R, Autti-Ramo I, Anttila H, Malmivaara A, Makela M. A systematic review finds that methodological quality is better than Methodology 
its reputation but can be improved in physiotherapy trials in childhood cerebral palsy. J Clin Epidemiol 2006 59:1 239-1 248 

MacPherson H, Thorpe L, Thomas K. Beyond needling - therapeutic processes in acupuncture care: a qualitative study nested Clinical 

within a low-back pain trial. J Altern Complement Med 2006 12:873-880 

Rapley T, May C, Heaven B, Murtagh M, Graham R, Kaner EF, Thomson R. Doctor-patient interaction in a randomised controlled Methodology 
trial of decision-support tools. 5oc Sci Med 2006 62:2267-2278 

Salas I. Methodology for reorganization of the cervical cancer program in Chile. Cancer Detect Prey 2006 30:38-43 Public health 

Sheik A, Baig K. An audit of the use and complications of glycoprotein llb/llla inhibitors in percutaneous coronary Clinical 
intervention against national UK standards. Cardiovasc Revasc Med 2006 7:237-239 

Sisk JE, Hebert PL, Horowitz CR, McLaughlin MA, Wang JJ, Chassin MR. Effects of nurse management on the quality of heart Clinical 
failure care in minority communities: a randomized trial. Ann Intern Med 2006 145:273-283 

Sturt J, Hearnshaw H, Farmer A, Dale J, Eldridge S. The Diabetes Manual trial protocol - a cluster randomized controlled trial Health 

of a self-management intervention for type 2 diabetes [ISRCTN0631541 1]. BMC Fam Pract 2006 7:45 promotion 

Sturt J, Taylor H, Docherty A, Dale J, Louise T. A psychological approach to providing self-management education for people Health 
with type 2 diabetes: the Diabetes Manual. BMC Fam Pract 2006 7:70 promotion 

Sturt J, Whitlock S, Hearnshaw H. Complex intervention development for diabetes self-management. J Adv Nurs 2006 Health 
54:293-303 promotion 

Treweek S, Sullivan F. How much does pre-trial testing influence complex intervention trials and would more testing make Methodology 

any difference? An email survey. BMC Med Res Methodol 2006 6:28 
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Table 1 Papers analysed by year of publication (Continued) 



2007 



Allegrante JP, Peterson MG, Cornell CN, MacKenzie CR, Robbins L, Horton R, Ganz SB, Ruchlin HS, Russo PW, Paget SA, Charlson ME. Methodology 
Methodological challenges of multiple-component intervention: lessons learned from a randomized controlled trial of 
functional recovery after hip fracture. HSS J 2007 3:63-70 

Armstrong N, Hearnshaw H, Powell J, Dale J. Stakeholder perspectives on the development of a virtual clinic for diabetes Health 

care: qualitative study. J Med Internet Res 2007 9:E23 promotion 

Ciuffreda D, Pantaleo G, Pascual M. Effects of immunosuppressive drugs on HIV infection: implications for solid-organ Clinical 

transplantation. Transpl Int 2007 8:649-658 

Garfein RS, Swartzendruber A, Ouellet LJ, Kapadia F, Hudson SM, Thiede H, Strathdee SA, Williams IT, Bailey SL, Hagan H, Golub ET, Methodology 
Kerndt P, Hanson DL, Latka MH. Methods to recruit and retain a cohort of young-adult injection drug users for the Third 
Collaborative Injection Drug Users Study/Drug Users Intervention Trial (CIDUS lll/DUIT) Drug Alcohol Depend 2007 91 Suppl 

1:S4-17 

Green J, Jacobs B, Beecham J, Dunn G, Kroll L, Tobias C, Briskman J. Inpatient treatment in child and adolescent psychiatry - a Clinical 
prospective study of health gain and costs. J Child Psychol Psychiatry 2007 48:1259-1267 

Hoddinott P, Pill R, Chalmers M. Health professionals, implementation and outcomes: reflections on a complex intervention Health 
to improve breastfeeding rates in primary care. Fam Pract 2007 24:84-91 promotion 

Lamb SE, Gates S, Underwood MR, Cooke MW, Ashby D, Szczepura A, Williams MA, Williamson EM, Withers EJ, Mt Isa S, Gumber A. Clinical 
Managing Injuries of the Neck Trial (MINT): design of a randomised controlled trial of treatments for whiplash associated 
disorders. BMC Musculoskelet Disord 2007 8:7 

MacPherson H, Schroer S. Acupuncture as a complex intervention for depression: a consensus method to develop a Clinical 
standardised treatment protocol for a randomised controlled trial. Complement Ther Med 2007 15:92-100 

May CR, Mair FS, Dowrick CF, Finch TL. Process evaluation for complex interventions in primary care: understanding trials Methodology 
using the normalization process model. BMC Fam Pract 2007 8:42 

Nardi R, Scanelli G, Corrao S, lori I, Mathieu G, Cataldi Amatrian R. Co-morbidity does not reflect complexity in internal Clinical 
medicine patients. Eur J Intern Med 2007 18:359-368 

Paul G, Smith SM, Whitford D, O'Kelly F, O'Dowd T. Development of a complex intervention to test the effectiveness of peer Health 
support in type 2 diabetes. BMC Health Serv Res 2007 7:1 36 promotion 

Paul GM, Smith SM, Whitford DL, O'Shea E, O'Kelly F, O'Dowd T. Peer support in type 2 diabetes: a randomised controlled trial Health 
in primary care with parallel economic and qualitative analyses: pilot study and protocol. BMC Fam Pract 2007 8:45 promotion 

Peters-Klimm F, Muller-Tasch T, Schellberg D, Gensichen J, Muth C, Herzog W, Szecsenyi J. Rationale, design and conduct of a Clinical 
randomised controlled trial evaluating a primary care-based complex intervention to improve the quality of life of heart 
failure patients: HICMan (Heidelberg Integrated Case Management). BMC Cardiovasc Disord 2007 7:25 

Power R, Langhaug L, Cowan F "But there are no snakes in the wood": risk mapping as an outcome measure in evaluating Methodology 

complex interventions. Sex Fransm Infect 2007 83:232-236 

Protheroe J, Bower P, Chew-Graham C. The use of mixed methodology in evaluating complex interventions: identifying Methodology 
patient factors that moderate the effects of a decision aid. Fam Pract 2007 24:594-600 

Protopopoff N, Van Herp M, Maes P, Reid T, Baz D, D'Alessandro U, Van Bortel W, Coosemans M. Vector control in a malaria Public health 

epidemic occurring within a complex emergency situation in Burundi: a case study. Malar J 2007 6:93 

Qian X, Smith H, Huang W, Zhang J, Huang Y, Garner P. Promoting contraceptive use among unmarried female migrants in Health 
one factory in Shanghai: a pilot workplace intervention. BMC Health Serv Res 2007 7:77 promotion 

Spillane V, Byrne MC, Byrne, M Leathern CS, O'Malley M, Cupples ME. Monitoring treatment fidelity in a randomized controlled Methodology 
trial of a complex intervention. J Adv Nurs 2007 60:343-352 

Turner DE, Helliwell PS, Woodburn J. Methodological considerations for a randomised controlled trial of podiatry care in Methodology 
rheumatoid arthritis: lessons from an exploratory trial. BMC Musculoskelet Disord 2007 8:109 

Wang HE, Abo BN, Lave JR, Yealy DM. How would minimum experience standards affect the distribution of out of hospital Clinical 
endotracheal intubations? Ann Emerg Med 2007 50:246-252 

Wiener-Ogilvie S, Pinnock H, Huby G, Sheikh A, Partridge MR, Gillies J. Do practices comply with key recommendations of the Clinical 
British Asthma Guideline? If not, why not? Prim Care Respir J 2007 16: 369-377 

2008 

Abelson JL, Khan S, Liberzon I, Erickson TM, Young EA. Effects of perceived control and cognitive coping on endocrine stress Clinical 
responses to pharmacological activation. Biol Psychiatry 2008 64:701-707 

Bosch-Capblanch X, Garner P. Primary health care supervision in developing countries. Trop Med Int Health 2008 13: 369-383 Other 

Byng R, Norman I, Redfern S, Jones R. Exposing the key functions of a complex intervention for shared care in mental health: Methodology 
case study of a process evaluation. BMC Health Serv Res 2008 8:274 
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Table 1 Papers analysed by year of publication (Continued) 



Carrasco MP, Salvador CH, Sagredo PG, Marquez-Montes J, Gonzalez de Mingo MA, Fragua JA, Rodriguez MC, Garcia-Olmos LM, 
Garcia-Lopez F, Carrero AM, Monteagudo JL. Impact of patient-general practitioner short-messages-based interaction on the 
control of hypertension in a follow-up service for low-to-medium risk hypertensive patients: a randomized controlled 
trial. IEEE Trans Inf Technol Biomed 2008 1 2:780-791 

de Salis I, Tomlin Z, Toerien M, Donovan J. Using qualitative research methods to improve recruitment to randomized 
controlled trials: the Quartet study. J Health Serv Res Policy 2008 1 3 Suppl 3:92-96 

de Salis I, Tomlin Z, Toerien M, Donovan J. Qualitative research to improve RCT recruitment: issues arising in establishing 

research collaborations. Contemp Clin Trials 2008 29:663-670 

Eisenberg ML, Elliott SP, McAninch JW. Management of restenosis after urethral stent placement. J Urol 2008 179:991-5 

Fairall LR, Bachmann MO, Zwarenstein MF, Lombard CJ, Uebel K, van Vuuren G, Steyn D, Boulle A, Bateman ED. Streamlining tasks 
and roles to expand treatment and care for HIV: randomised controlled trial protocol. Trials 2008 9:21 

Fearon KG. Cancer cachexia: developing multimodal therapy for a multidimensional problem. Eur J Cancer 2008 44:1 124-32 

Hawe P, Shiell A, Riley T. In response to Spillane V., Byrne M.C., Byrne M., Leathern C.S., O Malley M. & Cupples M.E. (2007) 
Monitoring treatment fidelity in a randomized trial of a complex intervention. Journal of Advanced Nursing 
60(3), 343-352. Important considerations for standardizing complex interventions. J Adv Nurs 2008 62:267 

Hendriks MR, Bleijlevens MH, van Haastregt JC, Bruijn FH, Diederiks IP, Mulder WJ, Ruijgrok JM, Stalenhoef PA, Crebolder HF,van Eijk 
JT A multidisciplinary fall prevention program for elderly persons: a feasibility study. Geriatr Nurs 2008 29:186-196 

Karamanidou C, Weinman J, Home R Improving haemodialysis patients' understanding of phosphate-binding medication: a 
pilot study of a psycho-educational intervention designed to change patients' perceptions of the problem and treatment. 

Br J Health Psychol 2008 1 3:205-2 1 4 

Klinkhammer-Schalke M, Koller M, Ehret C, Steinger B, Ernst B, Wyatt JC, Hofstadter F, Lorenz W. Implementing a system of 
quality of life diagnosis and therapy for breast cancer patients: results of an exploratory trial as a prerequisite for a 
subsequent RCT. Br J Cancer 2008 99:415-422 

Klinkhammer-Schalke M, Koller M, Wyatt JC, Steinger B, Ehret G, Ernst B, Hofstadter F, Lorenz W. Quality of life diagnosis and 
therapy as complex intervention for improvement of health in breast cancer patients: delineating the conceptual, 
methodological, and logistic requirements (modelling). Langenbecks Arch Surg 2008 393:1-12 

MacPherson H, Nahin R, Paterson C, Cassidy CM, Lewith GT, Hammerschlag R. Developments in acupuncture research: 
big-picture perspectives from the leading edge. J Altern Complement Med 2008 14:883-887 

MacPherson H, Thomas K. Self-help advice as a process integral to traditional acupuncture care: implications for trial 
design. Complement Ther Med 2008 16:101-106 

Narahari SR, Ryan TJ, Aggithaya MG, Bose KS, Prasanna KS. Evidence-based approaches for the Ayurvedic traditional herbal 
formulations: toward an Ayurvedic CONSORT model. J Altern Complement Med 2008 14:769-776 

Panella M, Marchisio S, Gardini A, Di Stanislao F. A cluster randomized controlled trial of a clinical pathway for hospital 
treatment of heart failure: study design and population. BMC Health Serv Res 2008 7:1 79 



Panella M, Brambilla R, Marchisio S, Di Stanislao F 
intervention. Stroke 2008 39:e186 



Reducing stroke in hospital mortality: organized care is a complex 



Patel VH, Kirkwood BR, Pednekar, S, Araya R, King M, Chisholm D, Simon G, Weiss H. Improving the outcomes of primary care 
attenders with common mental disorders in developing countries: a cluster randomized controlled trial of a collaborative 
stepped care intervention in Goa, India. Trials 2008 9:4 

Perkins D, Harris MF, Tan J, Christl B, Taggart J, Fanaian M. Engaging participants in a complex intervention trial in Australian 

General Practice. BMC Med Res Methodol 2008 8:55 

Sampson EL, Thune-Boyle I, Kukkastenvehmas R, Jones L, Tookman A, King M, Blanchard MR. Palliative care in advanced 
dementia; a mixed methods approach for the development of a complex intervention. BMC Palliat Care 2008 7:8 

Sifri ZC, Kim H, Lavery R, Mohr A, Livingston DH. The impact of obesity on the outcome of emergency intubation in trauma 

patients. J Trauma 2008 65:396-400 

Strong V, Waters R, Hibberd C, Murray G, Wall L, Walker J, McHugh G, Walker A, Sharpe M. Management of depression for people 
with cancer (SMaRT oncology 1): a randomised trial. Lancet 2008 372:40-48 
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researching and adopting medical emergency teams. Crit Care 2008 12:205 

Vaessen J, Todd D Methodological challenges of evaluating the impact of the Global Environment Facility's biodiversity 
program. Eval Program Plann 2008 31:231-240 
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guide. Clin Rehabil 2009 23:352-361 

Cagle JG, Kovacs PJ. Education: a complex and empowering social work intervention at the end of life. Health Soc Work 2009 

34:17-27 

Colombet I, Sabatier B, Gillaizeau F, Prognon P, Begue , Durieux P. Long-term effects of a multifaceted intervention to 
encourage the choice of the oral route for proton pump inhibitors: an interrupted time-series analysis. Qual Saf Health Care 
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Conboy L, Edshteyn I, Garivaltis H Ayurveda and Panchakarma: measuring the effects of a holistic health intervention. 

ScientificWorldJournal 2009 9:272-280 
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discharge process in an acute hospital. J Clin Nurs 2009 18:4 
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intervention improved randomization and informed consent in a randomized controlled trial. J Clin Epidemiol 2009 62:29-36 

Dumont A, Fournier P, Fraser W, Haddad S, Traore M, Diop I, Gueye M, Gaye A, Couturier F, Pasquier, JC, Beaudoin F, Lalonde A, 
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randomized trial of a multifaceted intervention to improve emergency obstetric care in Senegal and Mali. Trials 2009 10:85 

Dyas JV, Apekey T, Tilling M, Siriwardena AN Strategies for improving patient recruitment to focus groups in primary care: a 
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Farquhar MC, Higginson IJ, Fagan P, Booth S. The feasibility of a single-blinded fast-track pragmatic randomised controlled 
trial of a complex intervention for breathlessness in advanced disease. BMC Palliat Care 2009 8:9 

Holle R, Grassel E, Ruckdaschel S, Wunder S, Mehlig H, Marx P, Pirk O, Butzlaff M, Kunz S, Lauterberg J. Dementia care initiative in 
primary practice: study protocol of a cluster randomized trial on dementia management in a general practice setting. 

BMC Health Serv Res 2009 9:91 
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complex intervention to improve experienced continuity of care in patients with cancer. Br J Cancer 2009 100:274-280 

Mao J, Hou Y, Shang H, Wang H, Wang X, Zhao Y, Niu T, Cui J, Li G, Lin Q, Shi L, Jia X, Fan R, Wang B, Ruan J. Study on the 
evaluation of the clinical effects of traditional Chinese medicine in heart failure by complex intervention: protocol of 
SECETCM-HF. Trials 2009 1 0:1 22 

Mesotten D, Van den Berghe G. Clinical benefits of tight glycaemic control: focus on the intensive care unit. Best Tract Res 
Clin Anaesthesiol 2009 23:421 -429 

Murphy AW, Cupples ME, Smith SM, Byrne M, Byrne MC, Newell J. Effect of tailored practice and patient care plans on 
secondary prevention of heart disease in general practice: cluster randomised controlled trial. 2009 BMJ 339:b4220 
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Rickles D. Causality in complex interventions. Med Health Care Philos 2009 12:77-90 
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project in Germany. Eur Arch Psychiatry Clin Neurosci 2009 259 Suppl 2:S205-12 

Schroer S, MacPherson H, Adamson J Designing an RCT of acupuncture for depression-identifying appropriate patient 
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design and strategies to improve it. Palliat Med 2009 23:23-28 
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Siriwardena AN, Apekey T, Tilling M, Harrison A, Dyas JV, Middleton HC, Orner R, Sach T, Dewey M, Qureshi ZM. Effectiveness and Health 
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Thompson DR, Clark AM. Cardiac rehabilitation: into the future. Heart 2009 95:1897-1900 Clinical 
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2010 
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Rev 2010 CD007610 

Emsley R, Dunn G, White IR. Mediation and moderation of treatment effects in randomised controlled trials of complex Methodology 

interventions. Star Methods Med Res 2010 19:237-70 

Farquhar M, Higginson IJ, Fagan P, Booth S. Results of a pilot investigation into a complex intervention for breathlessness in Clinical 
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complex intervention. Qual Prim Care 2010 18:9-16 promotion 
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Korb-Savoldelli V, Sabatier B, Gillaizeau F, Guillemain R, Prognon P, Begue D, Durieux P. Non-adherence with drug treatment after Clinical 
heart or lung transplantation in adults: a systematic review. Patient Educ Couns 2010 81:148-154 

Langhorne P, Lewsey JD, Jhund PS, Gillies M, Chalmers JW, Redpath A, Briggs A, Walters M, Capewell S, McMurray JJ, Maclntyre K. Clinical 
Estimating the impact of stroke unit care in a whole population: an epidemiological study using routine data. J Neurol 
Neurosurg Psychiatry 2010 81:1301-1305 

Mayer S, Weisser M, Till H, Grafe G, Geyer C. Congenital myelomeningocele - do we have to change our management? Clinical 
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Rogers, A May, C Normalisation process theory: a framework for developing, evaluating and implementing complex 
interventions. BMC Med 2010 8:63 

O'Carroll, R. Dennis, M. Johnston, M Sudlow, C. Improving adherence to medication in stroke survivors (IAMSS): a randomised Clinical 
controlled trial: study protocol. BMC Neurol 2010 10:15 
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H. A complex intervention to support rest home' care: a pilot study. W Z Med J 201 0 1 23:41 -53 promotion 

Siddiqi K, Khan A, Ahmad M, Shafiq-ur-Rehman. An intervention to stop smoking among patients suspected of TB - Health 
evaluation of an integrated approach. BMC Public Health 2010 10:160 promotion 

Sollid SJ, Lossius HM, Nakstad AR, Aven T, Soreide E. Risk assessment of pre-hospital trauma airway management by Clinical 
anaesthesiologists using the predictive Bayesian approach. Scand J Trauma Resusc Emerg Med 2010 18:22 

Song J, Ratner ER, Wall MM, Bartels DM, Ulvestad N, Petroskas D, West M, Weber-Main AM, Grengs L, Gelberg L. Effect of an end- Health 
of-life planning intervention on the completion of advance directives in homeless persons: a randomized trial. Ann Intern promotion 

Med 2010 153:76-84 

Thompson A, Sullivan S, Barley M, Moore L, Rogers P, Sipos A, Harrison G. Effectiveness of a cognitive behavioural workbook for Health 
changing beliefs about antipsychotic polypharmacy: analysis from a cluster randomized controlled trial. J Eval Clin Pract promotion 
2010 16:520-528 



Thompson EA Alternative and complementary therapies for the menopause: a homeopathic approach. Maturitas 2010 

66:350-354 



Clinical 



Van Herck P, Vanhaecht K, Deneckere S, Bellemans J, Panella M, Barbieri A, Sermeus W. Key interventions and outcomes in joint Clinical 
arthroplasty clinical pathways: a systematic review. J Eval Clin Pract 2010 16:39-49 

Vanhaecht K, Sermeus W, Peers J, Lodewijckx C, Deneckere S, Leigheb F, Decramer M, Panella M. The impact of care pathways for Clinical 
exacerbation of Chronic Obstructive Pulmonary Disease: rationale and design of a cluster randomized controlled trial. 

Trials 2010 11:111 

Walker J, Fairley CK, Walker SM, Gurrin L C, Gunn JM, Pirotta MV, Carter R, Hocking JS. Computer reminders for Chlamydia Health 
screening in general practice: a randomized controlled trial. Sex Transm Dis 2010 37:445-450 promotion 

2011 

Barasa EW, English M. Economic evaluation of package of care interventions employing clinical guidelines. Trap Med Int Methodology 
Health 2011 16:97-104 

Barker M, Baird J, Lawrence W, Jarman M, Black C, Barnard K, Cradock S, Davies J, Margetts B Inskip H, Cooper C. The Southampton Health 
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Begh RA, Aveyard P, Upton P, Bhopal RS, White M, Amos A, Prescott RJ, Bedi R, Barton PM, Fletcher M, Gill P, Zaidi Q, Sheikh A. Health 
Experiences of outreach workers in promoting smoking cessation to Bangladeshi and Pakistani men: longitudinal promotion 
qualitative evaluation. SMC Public Health 201 1 11 :452 
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therapy intervention within the context of a clinical trial: a multimodal individualized treatment for patients with hip 
osteoarthritis. Phys Ther 201 1 91:1525-1541 

Bird L, Arthur A, Cox K. "Did the trial kill the intervention?" experiences from the development, implementation and Clinical 
evaluation of a complex intervention. BMC Med Res Methodol 201 1 11:24 

Boase S, Kim Y, Craven A, Cohn S. Involving practice nurses in primary care research: the experience of multiple and Methodology 

competing demands. J Adv Nurs 201 1 68:590-599 

Borglin G, Gustafsson M, Krona H. A theory-based educational intervention targeting nurses' attitudes and knowledge Clinical 
concerning cancer-related pain management: a study protocol of a quasi-experimental design. BMC Health Sen Res 201 1 

11:233 

Brady MC, Stott DJ, Norrie J, Chalmers C, St George B, Sweeney PM, Langhorne P. Developing and evaluating the Clinical 
implementation of a complex intervention: using mixed methods to inform the design of a randomised controlled trial of 
an oral healthcare intervention after stroke. Trial 201 1 12:168 



Braun SM, Wade DT, Beurskens AJ. Use of movement imagery in neurorehabilitation: researching effects of a complex 

intervention. Int J Rehabil Res 201 1 34:203-208 
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11:54 

Carter G, Britton B, Clover K, Rogers K, Adams C, McElduff P. Effectiveness of QUICATOUCH: a computerised touch screen Clinical 
evaluation for pain and distress in ambulatory oncology patients in Newcastle, Australia. Psycho-Oncology 201 1 
DOI:10.1002/pon.2020 

Charles JM, Bywater T, Edwards RT. Parenting interventions: a systematic review of the economic evidence. Child Care Health Health 
Dev 201 1 37:462-474 promotion 

Christie D, Hudson L, Mathiot A, Cole TJ, Karlsen S, Kessel A, Kinra S, Morris S, Nazareth I, Sovio U, Wong IC, Viner RM. Assessing the Health 
efficacy of the healthy eating and lifestyle programme (HELP) compared with enhanced standard care of the obese promotion 
adolescent in the community: study protocol for a randomized controlled trial. Trials 201 1 12:242 

Costantini M, Ottonelli S, Canavacci L, Pellegrini F, Beccaro M. The effectiveness of the Liverpool care pathway in improving Clinical 
end of life care for dying cancer patients in hospital. A cluster randomised trial. BMC Health Serv Res 201 1 11:13 

Dowding DW, Cheyne HL, Hundley V. Complex interventions in midwifery care: reflections on the design and evaluation of Methodology 
an algorithm for the diagnosis of labour. Midwifery 201 1 27:654-659 
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elements of cognitive behaviour therapy for psychosis: results of a novel type of subgroup analysis based on principal 
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Edwards ED, Mason BW, Oliver A, Powell CV. Cohort study to test the predictability of the Melbourne criteria for activation of Clinical 
the medical emergency team. Arch Dis Child 201 1 96:1 74-1 79 

Farquhar MC, Ewing G, Booth S Using mixed methods to develop and evaluate complex interventions in palliative care Methodology 

research. Palliat Med 201 1 25:748-757 

Farquhar MC, Prevost AT, McCrone P, Higginson IJ, Gray J, Brafman-Kennedy B, Booth S. Study protocol: Phase III single-blinded Clinical 
fast-track pragmatic randomised controlled trial of a complex intervention for breathlessness in advanced disease. 

Trials 2011 12:130 
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help? BMC Public Health 201 1 11 (Suppl 5): S8 

Gask L, Dowrick C, Salmon P, Peters S, Morriss R. Reattribution reconsidered: narrative review and reflections on an Methodology 
educational intervention for medically unexplained symptoms in primary care settings. J Psychosom Res 201 1 71:325-334 

Greenhalgh T, Campbell-Richards D, Vijayaraghavan S, Collard A, Malik F, Griffin M, Morris J, Claydon A, Macfarlane F. New models Health 
of self-management education for minority ethnic groups: pilot randomized trial of a story-sharing intervention. promotion 
J Health Serv Res Policy 201 1 1 6:28-36 

Gustafsson L, Nugent N, Biros L. Occupational therapy practice in hospital-based stroke rehabilitation? Scand J Occup Ther Clinical 
2011 DOI: 10.3109/11038128 
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11:63-74 

Hansen DG, Bergholdt SH, Holm L, Kragstrup J, Bladt T.Sondergaard J. A complex intervention to enhance the involvement of Clinical 
general practitioners in cancer rehabilitation. Protocol for a randomised controlled trial and feasibility study of a 
multimodal intervention. Acta Oncol 201 1 50:299-306 

Hirsch O, Keller H, Krones T, Donner-Banzhoff N. Acceptance of shared decision making with reference to an electronic library Health 
of decision aids (arriba-lib) and its association to decision making in patients: an evaluation study. Implement Sci 201 1 6:70 promotion 

James DM The applicability of normalisation process theory to speech and language therapy: a review of qualitative Methodology 
research on a speech and language intervention. Implement Sci 201 1 6:95 
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Kirkevold M, Bronken BA, Martinsen R, Kvigne K. Promoting psychosocial well-being following a stroke: Developing a Health 

theoretically and empirically sound complex intervention. Int J Nurs Stud 201 1 DOI: 1 0.1 01 6/j.ijnurstu promotion 

Lee MS, Choi TY.Kim Jl, Kim L, Ernst E. Acupuncture for treating attention deficit hyperactivity disorder: a systematic review Clinical 

and meta-analysis. Chin J Integr Med 201 1 17:257-260 

Lossius HM, Sollid SJ, Rehn M, Lockey DJ. Revisiting the value of pre-hospital tracheal intubation: an all time systematic Clinical 
literature review extracting the Utstein airway core variables. Crit Care 201 1 15:R26 

Maratos A, Crawford MJ, Procter S. Music therapy for depression: it seems to work, but how? Br J Psychiatry 201 1 199:92-93 Clinical 
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McQuay H. Evidence-based medicine: what is the evidence that it has made a difference? Palliat Med 201 1 25:394-397 Methodology 

Melis RJ, Teerenstra S,0lde Rikkert MG, Borm GF. Pseudo cluster randomization: balancing the disadvantages of cluster and Methodology 
individual randomization. Eva I Health Prof 201 1 34:151-163 

Milanese S. The use of RCTs in manual therapy- are we trying to fit a round peg into a square hole? Man Ther 201 1 Methodology 

16:403-405 

Morrison JJ, Mellor A, Midwinter M, Mahoney PF, Clasper JC. Is pre-hospital thoracotomy necessary in the military Clinical 
environment? Injury 201 1 42:469-473 

Ogilvie D, Bull F, Powell J, Cooper AR, Brand C, Mutrie N, Preston J, Rutter H. An applied ecological framework for evaluating Public health 
infrastructure to promote walking and cycling: the iConnect study. Am J Public Health 201 1 101:473-481 

Price S, Long AF, Godfrey M, Thomas KJ. Getting inside acupuncture trials-exploring intervention theory and rationale. BMC Methodology 
Complement Altern Med 201 1 1 1:22 

Sampson EL, Jones L, Thune-Boyle IC, Kukkastenvehmas R, King M, Leurent B, Tookman A, Blanchard MR. Palliative assessment Clinical 
and advance care planning in severe dementia: an exploratory randomized controlled trial of a complex intervention. 

Palliat Med 2011 25:197-209 

Schiapparelli P, Allais G, Rolando S, Airola G, Borgogno P, Terzi MG, Benedetto C. Acupuncture in primary headache treatment. Clinical 
Neurol Sci 201 1 32 Suppl 1 :S1 5-8 

Schroer S, Adamson J. Acupuncture for depression: a critique of the evidence base. CNS Neurosci Ther 201 1 17: 398-410 Methodology 

Senn B, Kirsch M, Sanz CC, Karlou C, Tulus K, De Leeuw J, Ringner A, Goossens GA, Cleary V. How cancer research could benefit Methodology 
from the Complex Intervention Framework: students' experiences of the European Academy of Nursing Science summer 
school. Eur J Cancer Care [Engl] 201 1 20:1-4 

Siddiqi N, Young J, House AO, Featherstone I, Hopton A, Martin C, Edwards J, Krishnan R, Peacock R, Holt R. Stop Deliriuml A Clinical 
complex intervention to prevent delirium in care homes: a mixed-methods feasibility study. Age Ageing 201 1 40:90-98 

Wallcraft J. The person in health care policy development. J Eval Clin Pract 201 1 17:347-349 Public health 

Young K, Bunn F, Trivedi D, Dickinson A. Nutritional education for community dwelling older people: a systematic review of Health 
randomised controlled trials. Int J Nurs Stud 201 1 48:751-780 promotion 



whether or not citation of these guidance documents or 
other key references reflects the content or quality of the 
papers listed, we report the proportions that did so. In all, 
31% (n = 64) of papers cited either the 2000 or 2008 guid- 
ance or both. Nearly a quarter (23%) cited Campbell 
et al.'s paper, 'Framework for design and evaluation of 
complex interventions to improve health [8] which ac- 
companied the 2000 guidance and 12.5% (n = 26) cited 
Craig et al.'s paper [1] which accompanied the launch of 
the 2008 MRC guidance. Other highly cited papers were: 
Campbell et al.'s 2007 paper 'Designing and evaluating 
complex interventions to improve health care' [9] (n = 16), 
Hawe, Sheill and Riley's 2004 'Complex interventions: how 
"out of control" can a randomised controlled trial be?' [10] 
(n=15) and Oakley et al.'s 2006 'Process evaluation in 
randomised controlled trials of complex interventions' 
[11] (n = 12). 43% (n = 90) of papers did not cite the guid- 
ance documents or any of the above key references. 

Intervention design 

Here we outline the challenges described by writers in 
deciding upon and standardising the content of inter- 
ventions which may include a number of components. 

The value of a theoretical understanding 

The MRC guidance advises that intervention design 
should be based on a theoretical understanding of how 



an intervention causes change. Some papers focused on 
the development of an explanatory framework or ration- 
ale to inform intervention design and evaluation. These 
included, for example, one aimed at identifying and dif- 
ferentiating the components of two approaches to acu- 
puncture (biomedical and traditional). These authors 
describe using a 'realist review' approach to develop an 
analytical framework for their review: 

'Its first step is to uncover or identify the essential or 
implicit theory or theories that underlie an 
intervention, that is, how the intervention is thought 
or meant to work and its expected impacts.' [12] 

Another research team described the process of devel- 
oping an optimal complex physical therapy intervention 
for patients with hip osteoarthritis 'in light of current 
knowledge and expert opinion' given the lack of under- 
standing about how individual components of the therapy 
affect the disease process [13]. In this case, the develop- 
ment of a theoretical framework meant collecting evi- 
dence to help understand the aetiopathogenesis and 
physical impairments associated with the condition. 

Others explained how they used existing models and 
theories to inform interventions and evaluation design. 
Drawing on previous studies, Borglin, Gustafsson and 
Krona [14] describe using the Theory of Planned 
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Behaviour to develop a series of workshops for nurses to 
improve pain management for cancer patients. The Nor- 
malisation Process Model [15] was used as a theoretical 
framework in two RCTs in maternity care and was reported 
to be of value in understanding organisational contexts into 
which new models of care are introduced [16]. 

...the use of this theoretical model will deepen our 
understanding of which factors contribute to the 
legitimacy of an intervention and thus the likelihood 
that it will be sustainable.' [16] 

Even if evidence is available, it may not be possible to 
predict which elements of an intervention will be accept- 
able to health care staff and patients, have the desired ef- 
fect and be sustainable. It may also be difficult to define 
exactly what will constitute the intervention: 

'...developing precise inclusion criteria for such 
complex interventions is more problematic, because 
by definition it is not clear a priori which mechanisms 
have to be in place in order to define an intervention 
as "collaborative care".' [17] 

Nonetheless, the authors report that developing a 
theoretical framework early in a study enables atten- 
tion to be focused on what needs to be done to plan, 
implement and sustain an intervention and what is less 
important. 

Standardisation and treatment fidelity 

Implementing any intervention in a standard format 
across sites is not straightforward but standardising 
complex, multiple treatment interventions, which may 
have a number of interacting components, is difficult 
and, some researchers argue, standardising the form of 
an intervention rather than its function may not be ap- 
propriate [18]. Two main challenges to standardisation 
were identified in these papers: on the supply side, the 
likelihood of variation in the delivery of services (e.g. 
[19]), and, on the demand side, the wide range of pa- 
tients' diagnoses, stages of disease, needs and prefer- 
ences (e.g. [20]). 

'Because of heterogeneity regarding settings, 
experiences, training, etc. and lack of standardisation, 
it is very difficult to compare different HPCTs 
[hospital palliative care teams]; hence the need for 
careful definition.' [21] 

'In the example given [...] of a Computer Decision 
Support System, is the intervention the software or 
the combination of the software and the staff working 
in the call centre?' [15] 



Attempting to standardise an intervention to meet the 
needs of researchers may lead to perverse outcomes: 

'The advantage of standardisation [in acupuncture 
interventions] must be offset against the disadvantage 
that such treatments, when obviously inadequate or 
inappropriate, cannot be modified, as would normally 
occur in routine clinical practice.' [22] 

A degree of flexibility in the design and implementa- 
tion of interventions was advocated by a number of 
writers with the aim of ensuring that interventions could 
be adapted to both local circumstances and to patients' 
needs. 

'...it is important to retain some flexibility, allowing 
adaptation of the intervention to the local context and 
ensuring the intervention can be tailored for 
individual OHC [oral health care] needs.' [23] 

As well as disparity in delivery, differences in the fre- 
quency of interventions and lack of a precise definition of 
the start of treatment were described (e.g. [24]). The MRC 
guidance [5] asserts that 'any variation in the intervention 
needs recording, whether or not it is intended, so that 
fidelity can be assessed in relation to the degree of stand- 
ardisation required by the study protocol'. Replicability 
would be compromised by undocumented variation. 

In order to record how implementation is carried out on 
the ground, the authors of one paper (on the topic of sec- 
ondary prevention of heart disease in general practice) 
suggest using a range of treatment fidelity procedures to 
monitor the intervention and to capture the processes in- 
volved. These procedures enhance validity and reliability 
with the aim of 'reducing errors in the interpretation of 
study outcomes and attributing outcomes directly to the 
effect of the intervention' [25]. Examples described include 
standardised training sessions, project manager observa- 
tion, quality assurance visits to practices during interven- 
tion implementation, use of a structured recall system, 
research nurse observation of general practitioners and 
practice nurses during intervention consultations and use 
of practice and patient care plans to document the process 
of intervention delivery. 

Intervention implementation 

To implement an intervention one must think at an 
early stage about who will be responsible for what and 
in what setting [5]. In the case of complex interventions, 
there may be a number of individuals, institutions or 
agencies involved across several sites. After an interven- 
tion has been trialled or evaluated (and depending on 
the outcome), consideration should be given to its sus- 
tainability and the ease with which it can be integrated 
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into usual service. In this section, we consider the 
challenges - ranging from the philosophical to the practical - 
identified by writers in implementing interventions. 

'Even when the concept of RRS [rapid response 
systems] is believed to be advantageous, the actual 
implementation entails overcoming a myriad of 
barriers: political, financial, educational, cultural, 
logistic, anthropological, and emotional.' [26] 

Structural and logistical obstacles may have an impact 
on effective implementation in the 'real world' where it 
is not always possible to control activities and outcomes. 

'Campus Watch has undergone many changes, both 
structural and functional, since it was introduced in 
2007; its evolution has not been guided by an 
overarching design and modifications have occurred 
for reasons that have not always been well 
documented.' [27] 

Staffing issues 

Those at the front line of 'delivering' an intervention 
may face time and resource difficulties or lack of buy-in 
with the aims of the intervention while there may be 
political and/or financial considerations further up the 
organisational hierarchy. The replication, regulation and 
sustainability of new practices in diverse teams across a 
number of sites can make heavy demands on staff who 
may experience competing priorities if they are also 
involved in data collection for evaluation purposes: 

'There was no systematic exploration of midwives' 
views of working in the models post RCT, or of the 
views of other stakeholders such as non-team 
midwives, managers and obstetric staff during or after 
completion of the team RCT, nor during the 
subsequent iterations of the team model. Therefore it 
is not possible to draw conclusions about why the 
original evaluated model was not sustained.' [16] 

'In ...the area where breastfeeding rates did not 
improve, health professional support for the project 
was weaker and relationships between midwives and 
health visitors were problematic' [28] 

'It is clear that teachers found it difficult to deliver the 
programme for a variety of logistic reasons (low 
morale, lack of support and competing priorities at 
school) and contextual reasons (difficulty teaching 
about sensitive issues, switching from their traditional 
teacher role, and lack of trust between pupils and 
teachers).' [29] 



Implementing an intervention uniformly may create 
difficulties for clinicians whose first aim is to provide the 
most effective care to patients. The papers present 
examples of treatment that deviated from the protocol 
because of decisions made by staff: 

'At least two control patients are known to have 
received more intensive physical therapy, i.e. muscle- 
strength training, than they would have otherwise. We 
believe that once the surgeons sensed that patients 
receiving intensive physical therapy were responding 
well, the surgeons were likely to have encouraged 
their patients to get more physical therapy, thus 
further diluting the impact of the intervention.' [30] 

Patient issues 

A number of issues relating to patients were raised by 
authors. These included patients' preferences and pa- 
tient/staff interaction, and recruitment and retention to 
trials. Studies about the treatment of chronic illness, for 
example, emphasised the role of patients (and carers) in 
active management of health conditions [31,32] Less 
positively, one paper reported that for a number of rea- 
sons 'despite initial willingness, after a few weeks some 
patients [suffering from psychosis] no longer wanted to 
receive therapy' [33]. A review on the topic of patients 
with medically unexplained symptoms reported that 
patients distrusted doctors regarding emotional aspects 
of their problems while doctors were concerned about 
encouraging patient dependence [34]. 

Those conducting trials reported that recruitment 
and retention of participants may be negatively affected 
if the intervention targets patients who are severely ill 
or who are hard to reach. Examples reported included 
patients with advanced dementia and their carers [35], 
those receiving palliative care [21] and young drug 
users [36]. In the first example, unbiased comparisons 
could not be made between intervention and control 
groups because of sample attrition [35]. In their consid- 
eration of the strengths and weaknesses of a before- 
after study design, Simon and Higginson [21] offer 
suggestions for strategies (including inclusion of a con- 
trol group in research design, time series approaches, 
and more robust outcome measures) to control and 
limit secular trends, bias and confounders. Garfein and 
colleagues [36] describe one method used to retain 
participants: 

'Given the anticipated difficulty in retaining young 
IDUs [intravenous drug users] for a longitudinal study, 
follow-up window periods were designed such that 
the need for high retention was balanced with the 
need for uniform intervals between the intervention 
and follow-up assessments.' 
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In evaluating an intervention aimed at high-utilising 
patients with medically unexplained symptoms, Lyles 
and colleagues [37] describe how they achieved their 
impressive retention rate of 98%: 

'Remunerating participants in recognition of their 
time commitment helped to maintain interest. 
However, consistent, clear communication from 
project staff and persistence in contacting participants 
were also important factors in enrolling and retaining 
subjects. We maintained a communication link with 
participants at intervals throughout the project.' 

Contextual characteristics 

Complex interventions, by their nature, are more likely 
than simpler ones to depend for their success on the 
context in which they are implemented [38]. Authors 
described the impact of structural, capacity, professional 
and political factors on their introduction. The most 
commonly cited contextual barrier to implementation 
was the organisational context. As one author put it: 

'The findings concur with previous studies, which 
suggest that organisational environment and culture, 
and client factors may influence occupational therapy 
practice.' [39]. 

Organisational context encompassed a wide range of 
elements from the parochial to the regional or national 
level and included organisational cultures, such as hier- 
archies and professional boundaries, staffing arrange- 
ments, social, geographical and environmental barriers, 
and the impact of other simultaneous organisational 
changes. The organisational context could either help or 
hinder the implementation of an intervention - or do 
both at the same time. 

'More attention should be given to the systems into 
which policies and complex interventions intervene. 
Particularly how the negative consequences of the 
environment, resource shortages, organisational 
change, competing demands and leadership affect an 
organisation's ability to effectively deliver an 
intervention.' [40] 

'The difficulties of delivering complex interventions in 
inner city areas are well known to clinicians, and 
might be attributed variously to low levels of social 
support, high levels of deprivation, and relative 
residential instability. Such contextual disadvantages 
remain a therapeutic challenge.' [33] 

Although the changing of long-term entrenched 
practices of physicians and other professionals is known 



to be a difficult task, problem solving in expanding 
cycles was able to affect such a change and produce an 
effective cervical cancer screening programme with no 
increase in financial resources.' [41] 

Another example of an organisational barrier to im- 
plementation was lack of support for what were seen as 
demanding projects. GP practice staff, for example, 
were thought to have few incentives for engaging in 
thinking through and developing complex new service 
arrangements: 

'Furthermore the external environment was not a 
sufficiently supportive context for the scope of the 
proposed shared care developments: it was seen as "a 
big project". '[42] 

Outcomes 

Having established what outcome(s) an intervention is 
aiming to achieve, researchers face challenges in design- 
ing tools to effectively measure outcomes, understanding 
'the length and complexity of the causal chains linking 
intervention with outcome' [5], explaining discrepancies 
between expected and observed outcomes, and captur- 
ing the long term characteristics of outcomes after a trial 
or study is concluded. 

Multidimensional outcome measures 

Outcomes are likely to be plural and multi-dimensional, 
spanning 'the spectrum from mortality, morbidity, dis- 
ability, to satisfaction and cost' [43] as 'restricting the 
success indicator to one single health or behavioural 
outcome leads to many unsolved questions about the 
success factors for, and barriers to, the effectiveness of 
the intervention' [44]. Clinical pathways are aspects of 
complex interventions that may demand outcomes be 
measured across many domains including clinical, ser- 
vice, team, process and cost [38]. As well as breadth, 
outcome measures must take time into account and may 
be designed for the short, medium or long term or all 
three. 

'Given this degree of complexity identifying a single 
primary outcome measure to capture the impact of an 
OHC [oral health care] intervention is problematic. 
We would anticipate that a multifaceted OHC 
intervention would impact upon a range of 
components including for example dental referrals, 
staff knowledge and patients' oral health.' [23] 

'It is therefore critical that the impact of new models 
of care are rigorously evaluated, considering outcomes 
for women and infants as well as outcomes for 
midwives and other maternity care providers.' [16] 
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Assessing outcomes 

Apart from difficulties in deciding upon measureable 
outcomes imposed by the complexity of interventions, 
writers noted that there is now an expectation that the 
bio-psycho-social aspects of interventions be measured 
as well as the clinical ones [45]. In palliative care, for ex- 
ample, patient experience is the primary outcome [46]. 
In general, it is argued that patient-centred outcomes, 
such as quality of life, as well as the views and experi- 
ences of staff should be taken into account. Some au- 
thors suggested that methods of measuring outcomes 
did not always capture the positive impact of an inter- 
vention and, in some cases, described their use of quali- 
tative data to measure patient experience (e.g. [47]). 

'The lack of an objective outcome was in contrast to 
subjective feedback from the study participants who 
felt that the intervention had produced a change in 
practice.' [48] 

'Reliance on empirical and societal defined outcomes 
often hides success in terms of participant defined 
outcomes.' [19] 

Establishing 'hard' outcome measures was seen to be 
difficult in particular fields where the success of an inter- 
vention does not necessarily equate with patient im- 
provement or survival. 

'The holistic approach of palliative care and its 
services causes some problems in defining clear 
outcomes and finding valid measurements.' [21] 

'There is a lack of an accepted primary outcome 
regarding the use of decision aids. Possible categories 
to classify measures of effectiveness are knowledge, 
decision process (e.g., satisfaction and participation 
preference), decision outcomes (e.g., has a treatment 
decision been made, adherence), health status, and 
economic measures.'[49] 

Some writers admitted that it was not possible to attri- 
bute the 'active ingredient' [4] of a complex intervention 
to a particular component of its design: 

'If this complex intervention does reduce mortality the 
relative contributions of education, PEWS [paediatric 
early warning system] and MET [medical emergency 
team] to clinical effectiveness is unknown.' [50] 

'In many cases, the effectiveness of training is more 
difficult to measure because a wide range of variables 
unrelated to the training intervention can mediate 
both the training process and the outcome. These 



variables need to be considered if it is to be 
established whether an outcome is due to the training 
intervention or other unrelated factors. For instance, 
variables related to the individual have been shown to 
mediate impact on outcomes like stress and burnout 
levels, and staff satisfaction.' [51] 

Evaluation 

The process of evaluating health service interventions oc- 
curs before, during and after implementation. In this sec- 
tion, we highlight some important issues raised by authors 
but do not systematically describe the many research 
designs which are the subject of the papers themselves. 

Formative and process evaluation 

The 2008 MRC guidance suggests that 'A mixture of 
qualitative and quantitative methods is likely to be 
needed, for example to understand barriers to participa- 
tion and to estimate response rates' [5] to assess the 
feasibility of an intervention. As noted above, qualitative 
data are increasingly recognised as 'an essential compo- 
nent of health services research [52], providing insights 
into the acceptability of interventions and their social 
consequences which cannot be measured by quantitative 
approaches. Formative evaluation - conducted to aid 
intervention design - can offer insights into the views 
and priorities of both patients and practitioners. 

'The key to the successful development of the 
complex intervention was the use of qualitative 
research that ensured that the intervention was based 
on data from interactions in ongoing trial recruitment 
appointments. Exploratory qualitative research of 
recruitment appointments in the Protect feasibility 
study showed that improvements to the presentation 
of study information increased rates of randomization 
from 30% to over 65%.' [53] 

Process evaluation is particularly important in multisite 
trials, 'where the "same" intervention may be implemented 
and received in different ways' [11]. 

'Neither quantitative nor qualitative approaches alone 
would provide an adequate insight into the 
implementation of the intervention across all three 
levels of care, from the perspective of all involved and 
capture the information needed in relation to both 
effectiveness and feasibility issues.' [23] 

Discussion 

Limitations 

The number and range of papers discussed here are 
not comprehensive given the search terms used and 
database searched and selection bias is therefore 
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possible. However, we feel that there is a large enough 
number included for our purposes. We conducted a 
content analysis rather than a systematic review which 
supported our aim of identifying aspects of complexity 
in health interventions, the fields in which they are 
implemented and the challenges experiences by 
researchers. 



Summary of results 

The literature on complex interventions is thick with 
descriptions of complex, challenging interventions, but 
thin on practical advice on how these should be dealt 
with. In the papers we surveyed, authors pointed to the 
practical value of theory in determining which features 
of an intervention and its context are likely to be im- 
portant in influencing outcomes and determining 
sustainability. They caution against attempting to too 
narrowly define and standardise the intervention, draw- 
ing on Hawe and colleagues lead (standardising on 'func- 
tion' rather than 'form') [10]. This also means having 
procedures in place to document what is actually done 
under the heading of 'the intervention'. 

The interaction between intervention and context is 
frequently emphasised, and one aspect of context which 
is highlighted in several papers is the people involved, 
including staff and patients themselves. The MRC guid- 
ance notes that complexity may derive from interaction 
between patient or recipient and provider. The implica- 
tion for implementation and evaluation is that (in the 
case of healthcare interventions) barriers at both levels 
should be considered and mitigated and, in the case of 
evaluation, relevant data collected. These barriers could 
also be built into the initial logic model driving the 
evaluation [54]. The papers also point to the wide range 
of contexts which have been considered as relevant, in- 
cluding professional boundaries and hierarchies, which 
do not often feature in descriptions of context, but are 
clearly relevant in some of these examples. In one study 
the specific recommendation is made that attention 
should be given to the systems into which complex in- 
terventions are placed [40]. In practical terms this may 
mean describing those systems in detail and at different 
levels and theorising on how they may affect the effect- 
iveness of implementation. 

Several studies point to a multiplicity of health and 
non-health outcomes as a source of complexity. In many 
of the papers which raise this as an issue, there is an im- 
plicit need for outcome measures - or a range of out- 
come approaches - capable of capturing outcomes 
across different dimensions and time scales. This may 
imply a move away from a focus on primary outcomes 
and a small number of secondary outcomes towards a 
much more multi-criteria form of assessment which 



acknowledges the multiple objectives of many complex 
interventions. 

Conclusions 

Implications 

The above comments may have implications for 
reporting of studies of complex interventions. The 
quotes suggest that more detailed reporting of informa- 
tion on outcomes, context and intervention is required 
for complex interventions. However, reporting guide- 
lines for quantitative studies may require further adap- 
tation to enable adequate explanation of complex 
interventions, and the contexts within which they were 
implemented. Defining and describing context, for 
example, may prove particularly challenging and, given 
the inherent flexibility in complex interventions them- 
selves, even defining the intervention may be difficult. 
Future revisions to reporting guidelines for both pri- 
mary and secondary research may need to take aspects 
of complexity into account to enhance their value to 
both researchers and users of research. 
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